Definition and some Properties of Information Entropy
نویسندگان
چکیده
In this article we mainly define the information entropy [3], [11] and prove some its basic properties. First, we discuss some properties on four kinds of transformation functions between vector and matrix. The transformation functions are LineVec2Mx, ColVec2Mx, Vec2DiagMx and Mx2FinS. Mx2FinS is a horizontal concatenation operator for a given matrix, treating rows of the given matrix as finite sequences, yielding a new finite sequence by horizontally joining each row of the given matrix in order to index. Then we define each concept of information entropy for a probability sequence and two kinds of probability matrices, joint and conditional, that are defined in article [25]. Further, we discuss some properties of information entropy including Shannon’s lemma, maximum property, additivity and super-additivity properties.
منابع مشابه
A Preferred Definition of Conditional Rényi Entropy
The Rényi entropy is a generalization of Shannon entropy to a one-parameter family of entropies. Tsallis entropy too is a generalization of Shannon entropy. The measure for Tsallis entropy is non-logarithmic. After the introduction of Shannon entropy , the conditional Shannon entropy was derived and its properties became known. Also, for Tsallis entropy, the conditional entropy was introduced a...
متن کاملTsallis Entropy and Conditional Tsallis Entropy of Fuzzy Partitions
The purpose of this study is to define the concepts of Tsallis entropy and conditional Tsallis entropy of fuzzy partitions and to obtain some results concerning this kind entropy. We show that the Tsallis entropy of fuzzy partitions has the subadditivity and concavity properties. We study this information measure under the refinement and zero mode subset relations. We check the chain rules for ...
متن کاملSome properties of the parametric relative operator entropy
The notion of entropy was introduced by Clausius in 1850, and some of the main steps towards the consolidation of the concept were taken by Boltzmann and Gibbs. Since then several extensions and reformulations have been developed in various disciplines with motivations and applications in different subjects, such as statistical mechanics, information theory, and dynamical systems. Fujii and Kam...
متن کاملSome Results Based on Entropy Properties of Progressive Type-II Censored Data
In many life-testing and reliability studies, the experimenter might not always obtain complete information on failure times for all experimental units. One of the most common censoring schemes is progressive type-II censoring. The aim of this paper is characterizing the parent distributions based on Shannon entropy of progressive type-II censored order statistics. It is shown that the equality...
متن کاملSome Results on Weighted Cumulative Entropy
Considering Rao et al. (2004) and Di Crescenzo and Longobardi (2009) studies, Misagh et al. (2011) proposed a weighted information which is based on the cumulative entropy called Weighted Cumulative Entropy (WCE). The above-mentioned model is a Shiftdependent Uncertainty Measure. In this paper, we examine some of the properties of WCE and obtain some bounds for that. In order to ...
متن کاملSOME PROPERTIES OF UNCERTAIN INTEGRAL
Uncertainty theory is a mathematical methodology for dealing withnon-determinate phenomena in nature. As we all know, uncertainprocess and uncertain integral are important contents of uncertaintytheory, so it is necessary to have deep research. This paperpresents the definition and discusses some properties of strongcomonotonic uncertain process. Besides, some useful formulas ofuncertain integr...
متن کامل